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Executive  Summary 


Lightning  strikes  have  caused  damage  reaching  hundreds  of  millions  of  dollars 
during  major  unmanned  launches,  and  lightning  could  lead  to  loss  of  life  on  shuttle 
launches.  Reliable  predictions  of  lightning  strikes  would  be  invaluable.  This  report  shows 
the  feasibility  of  using  artificial  neural  systems  (ANS)  for  making  predictions  of  cloud  to 
ground  lighting  strikes.  ANS  provides  a  tool  for  cost-effectively  building  lightning 
prediction  machines  with  advantages  over  standard  programming. 

ANS  designs  offer  some  potentially  useful  features.  First,  an  ANS  predictor  can  be 
incrementally  trained  for  new  levels  of  performance  without  starting  programming  from 
'scratch'  each  time  the  predictor  is  upgraded.  Second,  incremental  training  could  proceed 
in  the  field  reducing  costs  and  delays  of  modifications  while  improving  predictor  accuracy 
by  tailoring  it  to  site  conditions  (i.e.  topography,  etc.).  Third,  a  trained  ANS  provides  a 
ready-made  formula  for  constructing  fast  parallel,  distributed  processors.  Fourth,  the 
features  built  up  within  the  ANS  might  be  analyzed  for  clues  to  the  physical  processes 
underlying  the  partially  understood  phenomenon  of  lightning. 

Comparisons  are  made  of  the  performance  of  an  ANS  predictor  with  the  state  of  the 
art  lightning  prediction  using  a  wind  convergence  based  criterion  described  by  Watson  et 
al.  1987.  Using  the  total  area  divergence  of  the  winds  at  the  Kennedy  Space  Center,  one 
predicts  lightning  when  this  quantity  is  less  than  a  certain  negative  threshold.  An  ANS  was 
trained  to  wind  inputs  and  known  lightning  events.  Both  the  ANS  and  wind  total  area 
divergence  methods  were  applied  for  similar  conditions.  The  results  show  that  ANS 
lightning  prediction  has  achieved  performance  levels  comparable  to  the  previous  state  of  the 
art  as  shown  below. 

Comparison  of  this  Study's  Results  with  those  of  Watson  et  al,  (1987) 


Type  of  Measure 

This  Study 

Watson,  et  al. 

(1987) 

Probability  of  Detection 

0.41 

0.47 

False  Alarm  Ratio 

0.57 

0.56 

Critical  Skill  Index 

0.26 

0.29 

Given  the  complexity  of  lightning  strikes,  the  current  results  should  be  understood 
as  indicative  of  a  promising  initiative  and  canro*  he  considered  definitive.  Therefore, 
recommendations  for  future  work  are  made.  The  results  in  the  Table  were  obtained  using 
one  day's  raw  wind  speed  and  direction  data  as  t  dng  input  to  the  ANS.  A  different 
day’s  wind  data  were  used  for  testing  ANS  predictions.  Synthesis  of  different  data  types  is 
one  of  the  possibilities  of  employing  ANS.  It  is  expected  that  ANS  predictive  power  will 
increase  when  many  inputs  (e.g.  electric  field,  wind  divergence,  temperature,  humidity, 
satellite  cloud  altitudes,  and  radar  returns)  are  simultaneously  included  in  the  input. 
Predictive  power  should  also  improve  when  many  days'  data  are  used  in  training.  A 
program  of  recommended  research  and  development  tasks  is  given  for  advancing  the  state 
of  the  art  of  lightning  predictors  which  use  ANS. 
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The  concept  of  using  artificial  neural  systems  (ANS)  for  predicting  lightning 
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Dr.  James  Stark  Draper  and  Dr.  Donald  S.  Frankel  in  planning  meetings  with  Dr.  Robert 
McClatchey,  Dr.  Arnold  Barnes,  and  Dr.  James  Metcalf  of  the  Geophysics  Laboratory 
(GL).  Subsequently,  under  sponsorship  by  GL  under  Air  Force  Contract  No.  F19628-89- 
C-0012  the  pilot  study  reported  here  was  initiated. 
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and  other  individuals  of  CSR/Weather,  NASA  and  Air  Force  personnel  at  the  Kennedy 
Space  Center  provided  the  meteorological  data  needed  in  this  study.  The  extensive  and 
professional  help  of  Ms.  Barbara  J.  Levine  in  preparing  contract  and  report  materials  is 
gratefully  acknowledged.  Acknowledgement  is  given  to  Mr.  Randall  W.  Bergmann, 
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L  Lightning  &  Missile  Launches:  A  Pilot  Study 

A  lightning  strike  to  a  rocket,  either  on  a  launch  pad  or  during  boost  through  the 
troposphere,  could  result  in  a  devastating  loss.  Danger  to  rocket  launches  from  lightning 
first  came  into  prominence  when  the  Apollo  12  Saturn  V  vehicle  was  struck  twice,  at 
altitudes  of  2  and  at  4.2  km  (Brook  et  al.,  1970).  The  second  stroke  initiated  a  rapid 
tumbling  of  the  inertial  navigation  subsystem  which  would  have  terminated  the  mission  but 
for  the  lucky  fact  that  this  subsystem  was  not  in  use  at  that  time.  The  loss  of  a 
communications  satellite  atop  an  Atlas  Centaur  rocket  represented  a  loss  of  some  $160 
million  (Wilford,  1987;  Doherty  ,  1987).  There  is  also  danger  to  fueling  crews  and  other 
ground  assets.  This  situation  has  led  the  USAF,  NASA  and  NOAA  to  consider  various 
means  for  reducing  the  lightning  threat. 

1.1.  Coping  with  Lightning 

Both  structural  and  electrical  damage  may  occur  due  to  lightning  striking  a  rocket 
just  before  or  during  launch.  In  the  first  type,  the  rocket  body  and  its  ionized  gas  trail  (the 
rocket  exhaust)  acts  as  a  'stationary  discharge  channel'  in  which  excessive  currents  bum 
out  circuits  and  structural  pieces.  In  the  second  type,  rapidly  changing  magnetic  fields  due 
to  nearby  lightning  strokes  causes  EMP  (electromagnetic  pulse)  damage  to  circuits.  Either 
type  of  damage  can  be  reduced  by  several  approaches. 

Efforts  to  cope  with  the  potential  of  lightning  damage  appear  to  fall  into  one  of 
several  approaches; 

(1)  Prevention  of  Damage  when  Struck  -  Prevention  by  better  design, 
documented  by  Gabrielson  (1988),  can  be  attained  using  lightning  rods  and 
otherwise  heavily  shielded  installations.  In  1982  the  shuttle  launch  tower  was 
protected  from  a  29  kA  stroke  which  was  grounded  by  a  lightning  rod 
protection  system  (Anon,  1983). 

(2)  Avoidance  of  Being  Struck  -  Avoidance  methods  may  be  classed  in  either 
flexible  or  non-flexible  approaches.  Historical  averages  of  local  stroke  activity 
can  be  used  to  show  which  locales  ought  to  be  avoided.  Historical  averages  are 
of  no  help  when  dealing  with  an  existing  facility  such  as  The  Kennedy  Space 
Center  (KSC). 

'Avoidance'  could  be  aided  by  time-dependent  predictions  which  KSC 
operations  personnel  might  use  in  positioning  the  rocket  and  scheduling 
launches.  One  time-dependent  approach  is  the  adherence  to  an  inflexible  policy 
of  waiting  for  'perfect'  conditions  (clear  sky,  low  humidity,  etc.).  Limiting 
launches  to  'perfect'  conditions  unnecessarily  sacrifices  valuable  launch 
windows,  such  as  those  controlled  by  astronomical  events. 

A  more  flexible  approach  is  preferred.  One  can  imagine  the  value  of  high 
credibility  predictions  (e.g.  >70%  chance  of  a  strike  at  area  fi  j)  in  1  to  2  hours) 
which  vary  with  time  (i.e.  are  'flexible')  as  environmental  conditions  vary. 
This  type  of  approach,  were  its  credibility  proven,  would  provide  KSC 
operations  with  a  rational  basis  for  making  decisions  as  to  how  to  avoid 
damage. 
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Mr.  William  Jafferis,  a  NASA  program  manager,  emphasizes  the  need  few  lightning 
forecasting  over  the  coming  1/2  hr  to  3  hr  interval  at  KSC  (Fincher,  1988).  In  sum,  a 
method  for  providing  time-dependent  stroke  predictions  is  clearly  of  interest. 

1.2.  Approach  of  the  Pilot  Study 

The  investigation  of  a  time  dependent  prediction  method  using  artificial  neural 
systems  (ANS)  for  Kennedy  Space  Center  (KSC)  is  the  subject  of  this  pilot  study.  KSC  is 
a  particularly  challenging  launch  locale  for  protection  from  lightning  sir  e  it  is  one  of  the 
most  active  lightning  strike  areas  in  the  United  States  (Doherty,  1987).  Tne  attainment  of  a 
prediction  capability  that  is  comparable  to  the  current  state  of  the  art  using  ANS  is  the  goal 
of  this  study.  Success  in  this  effort  would  encourage  further  development  of  ANS 
techniques  for  building  lightning  predictors.  Two  considerations  recommend  the  use  of 
ANS  (often  termed  'neural  networks').  The  first  is  that  one  can  train  a  useful  neural 
network  even  though  our  understanding  of  lightning  physics  remains  incomplete.  The 
second  is  the  suitability  of  ANS  to  the  prediction  of  future  events.  These  two  topics  are 
dealt  with  in  the  next  two  paragraphs. 

The  extensive  data  sets  at  KSC  and  the  partial  knowledge  of  lightning  events  makes 
ANS  a  suitable  tool  for  building  a  lightning  predictor.  This  situation  is  well  described  by 
Caudill  (1989):  "Neural  Networks  are  extremely  valuable  for  the  situation  where  we  don't 
know  exactly  what  features  are  important ,  but  we  do  have  an  idea  of  some  likely  ones. 
When  this  situation  occurs,  we  can  shorten  the  training  needed  by  the  network  (sometimes 
substantially)  by  setting  up  a  preset  bias  toward  the  features  we  think  are  important.  This 
method  is  often  performed  when  the  network  designer  has  some  domain  knowledge  of  the 
problem  being  attacked,  and  it  generally  results  in  better  network  designs  and  much  faster 
training.  And  in  some  problems  the  network  designer  knows  nothing  at  all  about  the 
problem,  so  the  network  has  to  build  the  feature  detectors  from  scratch,  or  rather  from  a 
random  set  of  weights.  Such  a  network  could  work  very  well,  assuming  an  appropriate 
architecture  for  the  problem  is  at  hand,  although  it  may  take  longer  to  train  than  if  the 
designer  gives  it  a  head  start."  Therefore,  as  some  understanding  of  the  current  knowledge 
of  lightning  is  valuable  in  building  ANS,  we  discuss  the  phenomenology  of  lightning  in 
section  2.0. 

Weather  is  often  viewed  as  a  chaotic  time  series  with  underlying  'attractors'  or  a 
deterministic  'map'  conveying  the  immanent  meteorological  physics.  We  have,  in  the 
previous  paragraph,  conveyed  the  notion  that,  with  some  understanding  of  the  knowledge 
domain  under  discussion,  a  good  first  cut  neural  network  architecture  (nodes  and  link 
topology)  can  be  achieved.  Lapedes  and  Farber  (1987)  have  shown  the  value  of  ANS  used 
for  predicting  the  future  behavior  of  complicated  systems  which  appear  to  behave 
'chaotically'.  Examples  of  this  type  of  'deterministic  chaos'  are  turbulence  in  fluids, 
chemical  reactions,  plasma  physics  and  weather.  Lapedes  and  Farber  conclude  that 
nonlinear  ANS  perform  well  in  this  application  as  they  build  very  good  approximations  of 
the  underlying  maps.  Performing  well  means,  for  Lapedes  and  Farber,  that  the  ANS 
provided  'excellent  prediction  of  complicated,  "random,"  time  series'. 

A  final,  compelling  consideration  for  choosing  ANS  in  this  pilot  study  is  the 
evaluation  of  the  possibility  of  making  a  weather  data  analysis  tool  which  runs  on  'off-the- 
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shelf  hardware  (a  Macintosh  IIx)1.  With  a  suitable  software  interface,  the  design  of  which 
has  been  guided  by  meteorologists,  a  competent  meteorologist  (not  a  computer  scientist) 
would  be  able  to  use  this  data  analysis  tool  in  his/her  own  office  to  carry  out  research  and 
build  predictors  using  his  or  her  own  data. 

This  study  had  as  its  primary  goal  the  examination  of  the  feasibility  of  training  ANS 
for  the  prediction  of  lightning  strikes  at  KSC.  Secondarily,  some  features  of  a  prototype 
tool  for  the  use  of  meteorologists  who  wish  to  analyze  large  masses  of  data  were 
investigated.  Last,  recommendations  for  subsequent  effort  were  prepared. 

2.  Prediction:  State  of  the  Art 

The  state  of  the  art  of  predicting  lightning  strokes  is  based  upon  knowledge  of  the 
physics  of  charged  clouds  and  familiarity  with  meteorological  data  from  the  locale  under 
investigation.  Lightning  physics  reviews  are  available  (Williams,  1988;  Uman,  1988, 
1989).  Available  prediction  studies  are  described  which  will  be  used  to  benchmark  the 
results  of  this  study. 

2.1.  Physics  of  Lightning 

Cloud  to  ground  lightning  strokes  from  convection  cells  that  appear  as 
thunderclouds  (cumulonimbus)  are  considered  here  (see  Figure  2.1).  Snowstorms, 
sandstorms,  volcanic  eruption  clouds,  and  even  clear  air  conditions  have  produced 
discharges  (Gabrielson,  1988);  these  types  of  lightning  are  not  discussed. 

Vertical  winds  leading  to  the  collision  of  particles  with  a  consequent  charge 
separation  are  the  ultimate  source  of  electrified  clouds.  There  is  no  complete  theory  of 
lightning  discharges.  Many  physical  aspects  of  lightning  are  not  well  understood  and  are 
controversial  (Anon  ,  1989b).  Extensive  anecdotal  information  and  quantitative  collections 
of  related  meteorological  data  are  available.  An  understanding  of  the  general  behavior  of 
lightning  is  invaluable  in  designing  ANS  for  predictors  of  lightning  strokes. 

A  number  of  relevant  observations  about  the  physics  of  lightning  will  be  valuable 
for  this  study.  Thunderclouds  are  usually  isolated  'cells’  or  lines  of  cells  associated  with 
advancing  cold  fronts  (Uman,  1986).  The  isolated  cell,  shown  in  Figure  2.1,  follows  a 
typical  life  cycle:  cumulus,  mature,  and  dissipating.  As  sun-driven  thermals  rise,  humid  air 
cools  at  a  lapse  rate  of  3.3  F  per  1,000  feet  of  altitude  while  surrounding  dry  air  cools  at 
5.5  F  per  1,000  feet  of  altitude.  The  lower  cooling  rate  of  humid  air  results  from  heat 
released  as  water  vapor  condenses.  Under  these  unstable  conditions  the  humid  air 
continues  to  rise  until  the  atmospheric  temperature  profile  reverses  at  15  to  20  km  and  the 
cloud  top  flattens  out  in  its  characteristic  'anvil  profile'.  The  upward  motion  is  termed 
vertical  convection  and  the  cell  is  sometimes  called  a  'convection  center.'  While  central  to 
the  storm  development,  vertical  air  flow  is  not  measured  by  Doppler  radars  that  follow  the 
horizontal  flow  or  'convergence'  winds  (see  Figure  2.1).  Convergence  has  been  emerging 
as  a  key  parameter  in  lightning  forecasting.(Lopez  et  al.,  1986a  and  1986b). 

During  formation,  the  surface  (sometimes  termed  'low  level')  winds  'converge'  at 
the  bottom  of  the  cloud  (see  Figure  2.1)  and  a  cumulus  cloud  rapidly  rises  to  mature  as  a 
cumulonimbus  over  a  period  of  an  hour.  At  maturity,  precipitation  and  lightning 


1  Macintosh  is  a  registered  trademark  of  Apple  Computer,  Inc. 
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discharges  are  typically  most  plentiful  (Watson  et  al.,  1987,  1989a  and  b).  In  its  final 
stages  the  cloud  collapses,  strong  surface  wind  divergence  appears  and  the  cell  disperses  in 
about  an  hour.  Advancing  cold  fronts  will  bring  longer  precipitation  periods  because  a  line 
of  cells  may  pass  overhead.  It  is  expected  that  a  lightning  prediction  method  will  benefit 
from  distinguishing  between  cells  (which  evolve  overhead)  and  cold  fronts  which  rapidly 
pass  through  one’s  area.  These  two  regimes  are  for  calm  weather  (in  which  stationary  cells 
develop  in  one  area)  and  a  moving  weather  (front)  'flow  regime'.  At  KSC  the  moving  flow 
regime  further  subdivides  into  direction  (NE,  S E,  SW,  and  NW)  with  markedly  differing 
lightning  flash  results  (Holle  et  al.,  1988). 


Thunderclouds  contain  vertically  moving  air  columns  in  which  rising  and  falling 
particles  collide  leading  to  a  separation  of  charges  (Williams,  1988).  The  separated  charges 
build  up  in  restricted  regions,  shown  in  Figure  2.1.  As  charged  clouds  form  or  pass  over  a 
site,  changes  in  the  earth's  electric  field  will  occur.  These  can  be  routinely  monitored  using 
'field  mills.'  Lightning  is  the  discharge  of  the  charge  surplus  to  the  earth  or  to  other 
regions  within  the  cloud.  A  discharge  to  the  ground  is  initiated  by  downward  'stepped 
leaders’  which  ionize  channels  which  subsequently  carry  upward  'return  strokes.'  Return 
strokes  carry  much  larger  currents  than  the  stepped  leaders  and  are  the  principal  source  of 
the  flash  and  thunder.  Thunder  results  from  the  explosion  of  air  heated  by  the  return 
stroke.  Lightning  discharges  also  occur  within  clouds  (intra-cloud)  while  discharges 
between  clouds  (inter-cloud)  are  infrequent  (Uman,  1986).  In  the  temperate  zones  the 
preponderance  of  all  flashes  are  within  clouds. 

A  wide  range  of  specialized  instruments  is  used  to  collect  needed  meteorological 
data.  Surface  wind  convergence  is  deduced  from  anemometer  networks,  electric  field 
behavior  from  field  mills,  precipitation  from  weather  radar,  and  cloud  top  altitudes  from 
weather  radars  and  satellites.  Rapid  response  optical  detectors  have  recorded  lightning 
spectral  pulses  in  central  Florida  (Cundiff  et  al.,  1986).  Lightning  flashes  have  been 
counted  and  their  currents  measured  by  networks  of  detectors  intercepting  their 
electromagnetic  (EM)  radiation.  Transportable  lightning  EM  monitors  are  available,  such  as 


Honeywell's  LSZ-850  LSS  (Lightning  Sensor  System)  for  aircraft  protection  (Anon, 
1988).  Rashes  or  strokes  can  be  located  by  radio  direction  finders  (giving  rise  to  'sferics' 
data  bases)  and,  recently  and  most  importantly,  by  networks  of  wide-band,  magnetic 
direction  finders  or  time  of  arrival  systems. 

Modem  lightning  detection  and  display  systems  are  impressive  and  extensive, 
rendering  substantial  service  to  scientific  and  commercial  weather  programs.  A  French 
company.  Dimensions,  a  spin-off  of  the  French  national  institute  for  aerospace  research 
(ONERA),  produces  SAFIR  (Anon  1989c).  This  system  consists  sensors  which  are  nodes 
in  a  network  that  warns  of  thunderstorm  and  lightning  hazards  over  ranges  up  to  300  km. 
SAFIR  is  in  operation  at  the  European  Space  Center  in  Kourou,  French  Guiana.  In  the 
United  States,  a  company  specializing  in  such  devices  is  Lightning  Location  and 
Protection,  Inc.,  (LLP)  of  Tucson,  AZ.  The  R*SCAN  Company  participated  in  a  national 
lightning  detection  network  and  evaluated  a  time  of  arrival  system,  LPATS  (Lightning 
Position  and  Tracking  System)  for  cloud-ground  (CG)  reporting  (Lyons  et  al.,  1988). 
This  system  is  described  (Orville  et  al.,  1983)  and  its  scientific  output  is  discussed  in  many 
sources  (e.g.  Williams  and  Orville,  1988).  Atmospheric  Research  Systems,  Inc., 
developed  LPATS  and  operates  a  prototype  network  in  Rorida.  These  networks  provide 
accurate  records  of  strike  location,  time  and  strength  over  ranges  of  several  100  km. 

2.2.  The  Art  of  Lightning  Prediction 

Notwithstanding  the  limited  understanding  of  lightning,  its  dramatic  aspects  and 
potential  for  damage  have  led  to  many  attempts  to  predict  when  and  where  lightning  will 
occur.  It  is  worthwhile  to  briefly  discuss  two  topics.  First,  the  brief  review  of  the  physics 
of  thunderstorms  given  in  the  previous  section  leads  naturally  to  consider  the  measured 
parameters  which  might  have  value  for  predicting  lightning  strikes.  There  is  an  extensive 
body  of  work  on  thunderstorm  statistics  and  some  of  the  theories  advanced  for  predicting 
lightning  strikes.  These  provide  a  benchmark  forjudging  the  quality  of  predictions. 

2.2.1.  Predictive  Parameters 

Complementary  with  the  physics  of  thunderclouds  there  are  available  many 
observations  that  provide  some  ideas  on  how  to  build  a  prediction  technique.  Locale  and 
local  structures  have  long  been  known  to  'attract'  lightning  strikes.  Ancient  legend 
observes  that  sacred  locations  and  plants  favored  by  the  Gods  were  more  frequently  hit  by 
lightning  and  therefore  Zeus  of  the  thunderbolt  favored  the  oak,  an  observation  confirmed 
by  measurement  (Frazer,  1922).  Tall,  conductive  structures  (towers,  trees,  mountains, 
lightning  rods)  are  associated  with  frequent  lightning  strikes.  The  Empire  State  Building  is 
hit  20  to  25  times  each  year  (Anon,  1989a).  Modem  versions  of  such  locale  maps  of  the 
frequencies  of  lightning  strikes  or  thunder  days  are  used  to  estimate  the  a  priori  probability 
of  lightning  strikes  at  specific  locations  as  shown  in  Figure  2.2  taken  from  Uman  (1986). 
Early  'thunderstorm  day'  data  are  in  Anon  (1952)  while  recent  'flash  density’  maps  are 
given  in  Holle  et  al.  (1988). 

As  described  in  section  2.1,  the  thundercloud  cross  section  (Milan,  1963),  is  a 
kind  of  chimney  in  which  air  rises  vertically  carrying  smaller  particles.  Larger  particles 
falling  past  the  rising  panicles  create  static  charges  which  build  up  in  the  cloud.  At  ground 
level  air  'converges'  upon  the  base  of  the  thundercloud  chimney  and  this  wind  convergence 
is  a  precursor  of  thunderclouds  (Watson  et  al.,  1987),  where  the  'convective  growth'  is 
'rooted  in  the  boundary  layer'  in  Florida  (Watson  et  al.,  1989a).  The  taller  the  'chimney', 
the  more  intense  the  displacement  of  charge  by  stronger  updrafts.  The  taller  the 
thunderhead,  the  more  likely  the  occurrence  of  lightning  strikes.  This  has  led  to  the 


suggestion  that  satellite  IR  maps  used  to  measure  cloud  top  altitudes  would  be  useful  in 
lightning  prediction.  Overall  the  spatial  distribution  of  the  flashes  is  related  to  the  cloud 
charge  distribution,  the  motion  of  the  cloud,  and  the  inherent  spatial  variability  of  strikes 
from  a  common  charge  center  (Krider,  1988). 

The  thunderstorm  days  of  Figure  2.2  can  be  used  to  make  an  a  priori  time  averaged 
prediction  of  lightning  strikes.  Time  averaged  predictions  of  lightning  strokes  per  square 
mile  per  year  are  made  using  the  the  mean  number  of  thunderstorm  days  per  year,  Nt. 
Observations  show  that  the  number  of  lightning  strikes  (cloud  to  ground)  per  square  mile 
per  year  is  0.05  to  0.80  times  Nt  (Uman,  1986).  At  KSC,  where  Nt  =  90,  one  expects  a 
historical  rate  in  the  range  of  4.5  to  72  strikes  per  square  mile  per  year.  The  range  in  Nt  is 
modified  by  topography  (for  examples  see  Lopez  and  Holle,  1986).  A  historical  rate  is 
useful  for  choosing  sites  for  facilities,  but  is  not  useful  for  adjusting  rocket  launch 
operations  which  are  sensitive  to  weather  changes.  Other  parameters  must  be  taken  into 
account  such  as  local  time.  Lopez  and  Holle  (1986)  show  that  the  preponderance  of  KSC 
flashes  occur  from  1400  to  2000  local  time.  Locale  affects  lightning  strike  maps.  In  the 
KSC  region,  Holle  et  al.  (1988)  report  an  average  of  up  to  15  to  25  flashes  per  square  mile 
per  year  based  on  three  years  of  data. 


Figure  2.2  Map  of  Mean  Number  of  Thunderstorm  Days,  from  (Uman,  1986). 

The  construction  of  a  lightning  predictor  should  use  lightning  stroke  'precursors' 
based  upon  the  following  topological  and  meteorological  data: 

(1)  Flow  regime  (calm,  NE,  SE,  SW,  and  NW); 

(2)  Humidity  of  surface  air  mass; 

(3)  Locale  and  local  structure, 

(4)  Wind  convergence; 

(5)  Altitude  of  thundercloud  tops; 

(6)  Electric  fields  at  ground  level. 
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It  would  be  encouraging  if  one  or  more  of  these  meteorological  data  sets  were  used  in 
showing  an  ability  to  predict  future  lightning  strikes.  This  pilot  study  uses  wind  total  area 
divergence  data  to  lay  the  ground  work  for  more  sophisticated  networks  simultaneously 
using  several  types  of  meteorological  data  sets  (see  recommendations  in  Section  5.2.) 

2.2.2.  Predictive  Methods 

Prediction  of  lightning  strokes  has  been  of  two  types:  (a)  a  priori  predictions  based 
upon  historically  based,  time-averaged  behavior,  and  (b)  dynamic  predictions  which,  being 
based  upon  current  meteorological  data,  are  responsive  quantitative  predictions.  An 
example  of  an  a  priori  time  averaged  prediction  has  been  given  in  section  2.2.1  in  the 
estimation  of  the  annual  thunderstorm  days  at  KSC.  Recent  work  has  focussed  on  more 
time-responsive  predictions  using  larger  and  more  complex  data  sets  (e.g.  radar  reflectivity, 
wind  convergence).  Some  severe  storm,  thunderstorm,  and  lightning  prediction  projects 
are  listed  in  Table  2. 

Austin  (1985),  citing  many  of  the  workers  in  Table  2.1,  gives  a  guardedly 
optimistic  prognosis  for  forecasting  using  pattern-recognition  of  synoptic  data  sets. 
Watson  et  al.  (1987),  based  upon  their  'contingency  table',  describe  their  preliminary 
results  as  giving  a  1  in  2  chance  for  forecasting  lightning  based  upon  convergence  (i.e.  data 
gathered  from  synoptic  wind  measurement  networks).  Watson  et  al.  (1987)  define  a 
contingency  event  as  "a  sustained  drop  of  total  area  divergence  of  more  than  1.5xl(M  s-i 
for  more  than  10  minutes"  (see  also  Watson  and  Blanchard,  1984)  This  is  a  substantial 
improvement  over  prediction  methods  based  upon  historical  rates  which  do  not  respond  to 
the  immediately  previous  weather  conditions. 

Table  2, 1  Studies  of  Thunderstorm  and  Lightning  Relationships 


Individual(s)  * 

Nature  of  Project 

Hu  (1963) 

First  ANS  trainable  forecasting  system 

Battan  (1965) 

Associates  precipitation  amount  and  flash  count 

Austin  &  Stansbury  (1971) 

Located  lightning  by  precipitation  events  seen  on  radar 

Marshall  &  Radhakant  (1978) 

Lightning  indicated  using  radar  precipitation  maps 

Watson  &  Blanchard(1984) 

Divergence  and  convective  precipitation  in  Florida 

Chema  et  al.  (1985) 

Satellite  Divisible  data  for  thunderstorm  assessments 

Watson  et  al.  (1987) 

Presents  convergence/flash  contingency  table 

Holle  et  al.  (1988) 

Convergence/stroke  data  correlations  with  flow  regimes 

Lampru  (1989) 

Systematic  use  of  ANS  for  predicting  severe  storms 

Barnes  (1989) 

Prediction  of  lightning  triggered  by  a  rocket  aloft 

Barnes  &  Frankel  (1990) 

Study  of  ANS  trained  to  predict  lightning  precursors 

♦Two  papers  which  appear  to  be  relevant  but  of  which  copies  were  not  obtained  are  Bent 
and  Llewellyn  (1976)  and  Burke  (1990). 

Forecasters,  using  a  national  lightning  detection  network,  project  lightning  activity 
based  on  viewing  maps  overlaid  with  lightning  stroke  activity  which  is  seen  to  be  moving 
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with  storm  fronts  (Lyons  et  al.,  1989).  Fincher  (1988)  has  speculated  that  lightning 
prediction  will,  in  a  few  years,  give  flash  probabilities  by  locale  and  time  of  day.  For  the 
present  study,  the  method  of  Watson  et  al.  (1987)  for  predicting  lightning  strikes  as 
developed  for  the  KSC  region  will  provide  a  benchmark  against  which  to  compare  this 
program’s  results. 
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The  design  of  the  ANS  for  the  prediction  of  lightning  required  our  experience  with 
ANS,  sources  of  appropriate  data  sets,  and  an  understanding  of  the  behavior  of  neural 
systems  and  the  physics  of  thunderstorms.  Once  the  networks  were  trained,  appropriate 
and  accepted  evaluation  techniques  were  used  to  rank  networks  by  type  and  training  level. 
Lastly,  attention  was  given  to  die  design  of  the  man/machine  interface  which  users  must 
invoke  to  operate  the  planned  lightning  predictor. 

3.1.  Network  Modes  and  the  Uses  of  Data 

There  are  two  modes  involved  in  the  use  of  ANS:  the  training  mode  and  the  actual 
use  mode  (see  Figure  3.1).  In  the  training  mode,  available  data  are  applied  to  the  network, 
which  then  uses  learning  algorithms  to  make  the  desired  transformation  from  inputs  to 
outputs  by  adjusting  the  strength  of  the  dynamic  links  connecting  its  processing  elements. 


Verified  Data  (Inputs  &  Outputs  Known) 


Training 
Data  Set 


Untrained 

Network 


Network 

in 

Training 


Testing 
Data  Set 


Network 

in 

Evaluation 


Trained 

Network 


Lightning 

Predictions 


Figure  3.1  Use  of  Verified  (Well-known)  Data  to  Train  and  Test  an  ANS  for  Predicting 
Lightning  Strikes  Using  Subsequently  Obtained  New  Meteorological  Data. 


Testing  the  training  effectiveness,  before  actual  use,  is  part  of  the  training  mode.  In  the 
actual  use  mode,  the  network  no  longer  learns,  but  rather,  it  is  used  to  produce  outputs 
based  on  new  meteorological  (the  'met  data'  in  Figure  3.1)  inputs.  Training  periods  may 
be  interspersed  with  testing  periods  as  the  availability  of  new  met  data  permits. 


During  training  it  is  important  to  keep  in  mind  that  data  play  two  different  roles  in 
the  AN S  as  seen  in  Figure  3.1.  The  first  role  data  play  is  as  input  (stimuli).  Inputs  used 
during  training  should  be  those  which  are  expected  to  be  available  when  the  network  is  in 
actual  operation.  The  second  role  for  data  is  as  output  (responses)  during  training.  In  the 
backpropagation  learning  scheme,  these  outputs  are  referred  to  as  "target  values".  During 
training,  the  target  value  data  are  used  to  "teach"  the  network  the  output  response  desired 
for  a  given  input.  In  this  study,  two  different  types  of  data  were  chosen  for  use  as  target 
values  which  resulted  in  two  distinct  ANS  predictors.  First,  the  occurrence  of  high  electric 
field  was  used  as  target  value,  as  an  indicator  both  of  cloud  to  ground  strokes  (Christian  et 
al.,  1989)  and  as  a  warning  of  the  danger  of  triggered  lightning  (Uman,  1988).  Second, 
LLP  data  were  used  to  'teach'  the  network  to  recognize  cloud  to  ground  strikes. 

Therefore,  our  ANS  predictors  were  divided  into  two  groups  according  to  the 
'training  data'  chosen  (see  Table  3.1):  (1)  electric  field  predictors,  and  (2)  cloud  to  ground 
strike  predictors.  An  ANS  trained  to  predict  cloud  electrification  would  be  of  value  for 
quantifying  correlations  with  the  input  data.  On  the  other  hand,  the  dangers  cloud  to 
ground  strikes  present  to  ground  personnel,  fueling  operations,  etc.,  recommended  that  this 
study  concentrate  on  'training  data'  (i.e.  ANS  training  'target'  values)  consisting  of  cloud 
to  ground  strikes.  Therefore,  though  it  might  be  harder  to  attain,  an  ANS  trained  to  predict 
cloud  to  ground  strikes  received  the  emphasis  in  this  study. 

Table  3.1.  Some  Possible  Combinations  of  Network  Inputs  and  Training  Data 


INPUTS 

TRAINING  DATA 

Wind 

Electric  Field 

Wind 

LLP 

Wind, 

Electric  Field 

Electric  Field 

Wind, 

Electric  Field 

LLP 

Wind, 

Divergence 

Electric  Field 

Wind, 

Divergence 

LLP 

Wind, 

Divergence, 

Electric  Field 

Electric  Field 

Wind, 

Divergence, 

Electric  Field 

LLP 

Input  data  for  the  network  could  include  the  wind  data,  the  electric  field  data, 
nearby  LLP  data  and  'products'  of  these  or  other  data.  'Products’  refers  to  results  of 
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processing  the  raw  data  in  some  way  to  derive  a  presumably  superior  quantity.  One  such 
product  which  has  been  shown  to  be  of  value  is  the  wind  Total  Area  Divergence  (Watson  et 
al.,  1987).  This  product  has  been  shown  to  predict  50%  of  cloud  to  ground  strokes  at 
KSC.  The  first  ANS  we  tried  used  only  the  raw  wind  data  as  inputs.  Later  networks  will 
include  the  divergence  as  input 

3.2.  Data  Sets 

The  primary  meteorological  data  obtained  from  the  Kennedy  Space  Center  (KSC) 
are  the  measurements  of  the  Wind  Mesonet,  the  Launch  Pad  Lightning  Warning  System 
(LPLWS)  and  the  Lightning  Location  and  Protection  System  (LLP).  The  first  two  of  these 
systems  consist  of  arrays  of  sensors  covering  the  entire  KSC  area.  The  three  LLP  system 
sensors  are  located  outside  KSC  and  cover  a  much  larger  area,  of  which  KSC  is  a  part.  All 
of  these  data  have  been  recorded  and  archived  on  a  regular  basis  by  a  private  contractor, 
currently  Computer  Sciences  Raytheon,  Meteorology  Section  (CSR/Weather).  Volumetric 
radar  scans  and  satellite  imagery  for  the  KSC  area  exist,  but  their  use  is  beyond  the  scope 
of  this  pilot  project.  Table  3.2  summarizes  the  data  available  for  this  pilot  study. 

Table  3.2.  Weather  Data  Available  for  this  Study. 


Data  Availability 

Data  Rate 

Wind  Mesonet 

Wind  Direction 

12,  30,  35,  54,  60,  162, 
204,  275,  295,  394,  492 

feet 

5  minute 

average 

Wind  Speed 

30,  35,  54,  60,  162,  204, 

275,  295,  394,  492  feet 

Temperature 

6,  12,  54,  162,  204,  492 

feet 

Dew  Point 

6,  12,  54,  162,  204,  492 

feet 

Barometric  Pressure 

6  feet 

Humidity 

6  feet 

LPLWS 

Electric  Field 

31  sensors  at  ground  level 

3-5  samples 

at  10  Hz 

LLP 

Lightning  Flash 
Latitude,  Longitude 

Southern  Florida 

irregular 
7500/hr  max. 

The  Wind  Mesonet  consists  of  an  array  of  sensors  mounted  at  various  altitudes  on 
utility  poles  or  other  available  structures.  These  sensors  measure  wind  speed,  wind 
direction,  dew  point,  humidity,  barometric  pressure  and  temperature.  Most  of  the  53  speed 
and  direction  sensors  are  at  12  and  54  feet  above  the  ground.  Temperature  is  measured 
primarily  at  6  feet  and  54  feet.  These  wind  data  are  measured  several  times  per  second,  but 
only  the  five  minute  average  values  are  available  in  the  archives. 
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The  hypothesis  upon  which  the  lightning  prediction  methods  used  as  benchmarks 
here  are  based  is  that  thunderstorms  induce  dramatic  changes  in  wind  flow  patterns.  The 
value  of  wind  data  can  be  appreciated  by  considering  the  work  of  Watson  et  al  (1987).  In 
their  work,  a  "product"  of  raw  wind  data  has  been  derived  and  shown  to  have  value  in 
predicting  lightning  strokes.  This  product  is  called  the  Total  Area  Divergence'.  It  is  a 
measure  of  the  rate  at  which  wind  is  flowing  out  of  the  KSC  area.  The  relationship  between 
the  Total  Area  Divergence  (i.e.  the  wind  field)  and  thunderstorm  electrical  activity  (i.e. 
electric  fields)  is  schematically  illustrated  in  Figure  2.1.  In  this  figure  the  surface  winds  are 
shown  flowing  in  (a  negative  Total  Area  Divergence)  toward  the  base  of  the  thundercloud. 
This  air  mass  subsequently  rises  vertically  leading  to  charge  separation  and  charged  regions 
which  lead  to  the  vertical  electric  fields  shown  in  Figure  2.1. 

At  this  date,  total  Area  Divergence  is  the  best  single  lightning  predictor  available  at 
KSC  at  this  time.  Our  results  are  compared  with  those  of  Watson  (1989c)  who  provided  us 
with  data  tape  containing  his  calculations  of  the  Total  Area  Divergence  for  the  period  we 
have  studied.  We  have  recommended  that  Total  Area  Divergence  be  among  the  inputs  to  the 
version  update  of  this  ANS. 

The  LPLWS  system  consists  of  31  sensors,  often  referred  to  as  ’field  mills',  which 
measure  the  vertical  component  of  the  local  electric  field.  If  properly  placed  away  from 
metal  structures  on  the  ground,  the  data  from  these  sensors  should  give  a  good  indication 
of  the  existence  of  cloud  electrification  overhead.  Cloud  electrification  data  have  two 
values.  First,  they  provide  a  near-term  warning  of  the  danger  of  cloud  to  ground  strokes. 
Second,  they  are  an  indication  that  aircraft  and  rockets  launched  into  those  clouds  may 
trigger  lightning.  When  examined  in  detail  and  compared  with  each  other,  these  data  can 
also  be  used  to  locate  nearby  cloud  to  ground  lightning  flashes. 

The  LPLWS  sensors  sample  the  electric  field  at  10  Hz,  but  since  the  installation  of 
the  Cyber  860  computer  system  at  KSC  in  1985,  only  3-5  samples  per  second  have  been 
archived.  Even  at  the  reduced  data  rate,  the  volume  of  LPLWS  data  makes  it  impractical  to 
store  all  of  the  data  indefinitely.  Periodically,  the  volume  of  stored  data  is  reduced  by  KSC 
personnel.  All  data  are  discarded  except  during  periods  beginning  three  hours  before  and 
ending  one  hour  after  the  issuance  of  a  Meteorological  Watch  Advisory.  During  a  visit  to 
KSC  in  February,  1989,  it  was  learned  that  the  last  time  this  had  been  done  was  May, 
1988.  Therefore,  as  24  hour  data  were  still  available  after  May  1988,  data  during  the 
succeeding  summer,  in  July  1988,  were  used  in  this  study. 

The  LLP  data  are  derived  from  three  magnetic  direction  finders  which  respond  to 
cloud  to  ground  lightning  strokes.  The  strokes'  latitude  and  longitude  are  determined  by 
triangulation.  These  data  cover  most  of  southern  Florida  and  capture  an  estimated  80%  of 
all  flashes  (MacGorman  and  Rust,  1988,  Barnes  and  Metcalf,  1988).  The  LLP  system 
does  not  respond  to  intra-cloud  lightning,  nor  to  strokes  which  have  a  large  horizontal 
component  in  their  path  to  ground. 

After  consultation  with  NASA  personnel,  the  Air  Force  Air  Weather  Service  and 
CSR/Weather,  a  two  week  period  was  chosen  during  which  there  were  archived  data  of  all 
three  types  for  days  when  there  was  thunderstorm  activity  and  for  days  when  there  was 
none.  This  period  was  July  12  to  July  27,  1988.  On  some  of  the  data  tapes,  these  days 
were  denoted  by  their  Julian  Day  sequence  number:  88194  to  88214.  In  the  Julian  system, 
January  1,  1988,  would  be  88001  and  December  31,  1988,  would  be  88366. 
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3.3.  Preprocessing 

As  with  any  project  which  involves  analysis  of  field  data  from  another  facility, 
significant  effort  was  required  to  edit  the  data  and  put  it  into  the  proper  form  for  analysis. 
The  cooperation  and  able  assistance  of  CSR/Weather,  NASA  and  Air  Force  personnel  were 
essential  to  the  completion  of  this  phase  of  the  project. 

The  Wind  Mesonet  data  and  the  LLP  data  can  be  obtained  from  KSC  on  tape  in 
ASCII  (American  Standard  Code  for  Information  Interchange)  format.  In  this  form,  the 
data  can  be  examined  by  any  text  editing  software.  The  LPLWS  data  are  voluminous.  In 
ASCII  format,  a  single  day's  data  occupy  eleven  9-track  tapes.  On  the  other  hand,  in 
binary  form  (computer  internal  representation)  the  storage  of  one  day's  data  requires  only 
one  tape.  Binary  format  is  thus  the  form  of  choice  for  the  LPLWS  data.  Of  course,  in  this 
form,  the  data  cannot  be  examined  by  word  processing  software. 

Use  of  the  LPLWS  data  was  complicated  by  several  factors.  The  least  difficult  of 
these  to  deal  with  was  the  fact  that  the  Cyber  computer  with  which  they  were  processed  at 
KSC  uses  12-bit  bytes  and  60  bit  words,  while  the  Macintosh  uses  8-bit  bytes  and  16  or  32 
bit  words.  Much  more  difficult  was  the  fact  that  the  Cyber  computer  system  inserts  a 
variable  number  of  housekeeping  words  at  various  (unpredictable)  points  in  the  data  on 
tape.  Failure  to  allow  for  these  words  results  in  incorrect  reading  of  the  data. 

The  KSC  weather  data  are  stored  on  9-track  magnetic  tapes.  The  data  are  not 
arranged  in  the  standard  tab-delimited  form  which  the  LightningLynx© 1  system  requires  its 
data  to  have.  Therefore,  stand-alone  applications  have  been  developed  which  convert  from 
binary  to  ASCII  form  as  necessary  and  reformat  the  data  into  the  tab-delimited  form.  To 
facilitate  their  use  the  data  processing  system  includes  a  Qualstar  model  1054  9-track  tape 
drive.  This  model  is  equipped  with  a  standard  SCSI  (Small  Computer  System  Interface) 
connector  which  allows  direct  transfer  of  tape  resident  data  to  the  Macintosh  IIx  internal 
disk  drive.  The  9-track  tape  drive  and  the  conversion  applications  with  the 
LightningLynx©  ANS  system  comprise  the  Geophysics  Laboratory  lightning  predictor 
system. 


3.4.  Wind/Lightning  Networks 

The  ANS  networks  used  are  three-level  backpropagation  designs  as  described  in 
Rumelhart  and  McClelland  (1986).  The  number  of  input  nodes  is  determined  by  the 
arrangement  of  the  types  and  amount  of  meteorological  data  sets  chosen  for  training  the 
network.  In  this  study  106  input  nodes  are  typically  used.  The  hidden  layer  nodes  are 
minimized  -  within  the  constraint  that  network  performance  is  not  critically  limited.  Lastly, 
64  output  nodes  were  used,  corresponding  to  16  lightning  strike  areas  (which  are  termed 
'tiles'  here)  each  with  4  time  intervals  for  prediction.  At  KSC  there  are  16  square  tiles  each 
of  which  is  4.58  nautical  miles  on  a  side.  The  total  field  covered  by  the  16  tiles  is  21 
nautical  miles  on  a  side. 

The  networks  are  designed  to  be  trained  to  the  correlations  among  a  variety  of 
meteorological  events.  Two  types  of  networks  were  studied:  (1)  surface  winds  to  electric 
fields  (E  Fields),  and  (2)  surface  winds  to  lightning  strokes.  The  data  used  have  a  direct 


1  "LightningLynx®"  is  a  copyright  of  KTAADN,  INC.,  reserved  1990 
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effect  on  the  network  layout,  i.e.  the  number  of  input,  hidden,  and  output  nodes.  Because 
of  the  meteorological  sensor  arrangement  at  KSC,  the  networks  used  here  typically  possess 
106  input  nodes  as  determined  by  the  wind  tower  grid.  They  possess  a  varying  number  of 
hidden  nodes  as  determined  by  the  performance  requirements.  Lastly,  these  networks  use 
64  output  nodes  corresponding  to  the  4  time  windows  for  the  16  areas  Coles’)  defined  for 
the  KSC  region  in  this  study,  see  Figure  3.2. 


PREDICTION  TIMES  (MINUTES  FROM  NOW): 


Figure  3.2  Tiles  and  training  periods  used  in  this  study.  One  network  output  node  is 
assigned  to  each  of  these  64  tiles. 

ANS  training  reduces  output  errors  (defined  as  the  difference  between  actual  and 
required  responses)  by  a  steepest  descent  method.  Training  proceeds  in  a  sequence  of 
'passes'  each  of  which  starts  from  the  level  of  learning  achieved  during  the  previous  pass. 
The  metric  used  in  this  method  is  a  summary  'pass  error’,  defined  here. 

Pass  Error  =  Er  { network  output  -  target  value }  ^  (3.1) 

where  the  summation  ranges  over  all  data  samples  and  all  output  nodes.  Generally,  the 
pass  error  decreases  toward  an  asymptotic  level  as  the  network  approaches  its  maximum 
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performance.  If  better  performance  is  required,  networks  with  more  hidden  nodes  can  be 
formed  and  trained. 

The  first  ANS  constructed  used  the  raw  wind  speed  and  direction  as  inputs  and  the 
electric  field  as  output.  For  each  wind  sensor,  the  wind  speed  and  direction  were  used  to 
generate  two  network  inputs:  the  wind  components  in  the  north  and  east  directions.  This 
network  was  trained  to  predict  electric  field  values  exceeding  500  V/m  at  the  ground.  Data 
used  were  from  July  18,  1988.  An  important  result  of  working  with  this  combination  of 
data  was  that  a  three  layer  backpropagation  network  with  fewer  than  70  hidden  layer  nodes 
would  not  converge  to  a  stable  state,  while  one  with  70  hidden  nodes  would  leam  stably. 

The  number  of  hidden  nodes  is  usually  felt  to  represent  the  'features'  in  the  physical 
process  under  study.  That  is  there  appear  to  be  a  number  of  approximately  70  features 
connecting  the  wind  field  with  the  electric  field.  Several  approaches  might  be  used  to 
reduce  the  number  of  hidden  nodes.  For  example,  if  the  wind  divergence  is  more  directly 
connected  to  the  electric  field,  then  use  of  divergence  might  usefully  reduce  input  and 
hidden  nodes  and  reduce  the  training  time  required. 


3  Layer  Network 
70  Hidden  Nodes 
Wind  Data  Input  from  7-18-88 


Figure  3.3  Training  History  for  a  106/70/64  network  using  wind  as  input  and  electrical 
field  as  target  value.  As  Pass  Error  Approaches  a  Constant  Value  Training  is  Completed. 


The  pass  error  history  for  a  106/70/641  network  is  shown  in  Figure  3.3.  Here 
history  is  given  in  the  number  of  network  training  passes  the  network  underwent.  On  the 
Macintosh  IIx  each  pass  required  about  4.5  minutes.  The  overall  decrease  in  pass  error  is 
indicative  of  continued  improvement  of  the  network  performance  with  each  succeeding 
pass.  Note  that  the  network  training  was  stopped  at  pass  126  for  testing  at  one  level  of 
performance  and  later  resumed  to  achieve  better  performance  (i.e.  lower  pass  error).  This 
shows  that  predictive  ANS  may  be  intermittently  trained  at  higher  performance  levels  as 
new  weather  history  is  presented.  When  training  with  fewer  hidden  nodes,  even  as  many 


1  This  terminology  is  "input  node  number/hidden  node  number/output  node  number" 
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as  65,  oscillations  in  the  pass  error  were  seen  while  no  substantial  reduction  or 
improvement  in  network  performance. 


The  second  ANS  type  used  the  wind  data  for  input  as  before,  but  used  LLP  data  of 
cloud  to  ground  lightning  strokes  to  assign  target  values.  Since  very  few  cloud  to  ground 
strokes  occurred  on  July  18,  1988,  data  from  July  24  and  25  were  used  for  training  these 
networks.  Remarkably,  much  smaller  networks  could  learn  stably  when  LLP  data  were 
used  for  assigning  target  values.  The  smallest  such  net  had  only  6  hidden  layer  nodes. 

The  training  and  evaluation  of  a  large  number  of  networks  was  required  to  reduce 
the  results  to  those  given  here.  A  major  improvement  in  the  wind  field  to  lighming  strike 
ANS  predictor  performance  is  expected  when  larger  sets  of  weather  (numbers  of  days  for 
the  same  weather  regime)  are  used.  Another  improvement  is  expected  when  network 
layouts  include  several  input  data  types  simultaneously  (e.g.  wind,  electric  field,  satellite 
cloud  top  altitudes,  and  others). 

3.5.  Network  Interface 

A  network  interface,  sketched  in  Figure  3.4.,  helps  evaluating  alternative  layouts 
for  ANS  predictors.  Users  of  the  projected  system  must  deal  with  several  complex  entities: 
extensive  data  bases,  training  and  retraining  large  ANS  and  the  evaluation  of 


performance  using  accepted  statistical  concepts.  These  jobs  are  handled  on  a  Macintosh  IIx 
desktop  computer  through  a  comprehensive  man/machine  interface  prepared  in  this  study. 
The  interface  is  a  screen  presentation  system  which  provides  the  following  features: 


(1)  Convenient  program  interaction  through  the  use  of  windows,  menus  and  mouse, 

(2)  Direct  access  to  large  data  sets, 

(3)  Creation,  training  and  retraining  of  alternative  ANS  designs,  and 

(4)  Presentation  of  ANS  predictions  with  standard  evaluation  measures. 
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This  lightning  prediction  system  is  called  LightningLynx©  and  it  is  built  on  the  Macintosh 
IIx  computer  with  a  flexible  user  screen  (see  Figure  3.4).  A  'bare  bones'  (Version  1 .0)  of 
the  interface  is  provided.  The  LightningLynx©  interface  adheres  to  Macintosh  standards 


4  File  Edit  Map  status  Net  Run 


Lightning  Prediction  Status 


Prttffcttoni 


Network 


3:02:26  PM 


Figure  3.5.  Lightning  Prediction  Screen  Display  with  Its  Windows  as  the  User  Sees. 


via  a  conventional  Macintosh  windowing  system.  The  user,  with  keyboard  and  mouse, 
can  gain  access  to  data  files,  networks  and  evaluation  methods  and  then  initiate  training  or 
testing  runs.  The  user,  through  the  interface,  can  request  data  files  of  the  following  types: 

(1)  Meteorological  data  sets  (wind,  electrical  fields,  etc.), 

(2)  Networks  (general  types  and  specific  trained  networks),  and 

(3)  Network  performance  output  files 

The  interface  software  allows  the  user  to  create  new  networks  for  training. 
Networks  may  be  saved  on  magnetic  media  at  any  stage  of  training  through  a  menu  item. 
Catalogues  of  ANS  predictor  systems  can  be  built  for  various  weather  regimes  at  a 
particular  site.  Saved  networks  may  be  recalled,  again  through  a  menu  item,  and  trained 
further  or  used  as  is  for  lightning  prediction.  It  is  expected  that  user  comment  will  aid 
further  interface  development 
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4.  Network  Scoring 

Appropriate  and  accepted  evaluation  techniques  are  used  to  rank  the  performance  of 
the  trained  ANS  used  in  lightning  prediction.  These  are  presented  and  applied  to  a  range  of 
trained  ANS  predictors  in  the  following  sections. 

4.1.  Evaluation  Method 

Evaluation  of  networks  in  various  stages  of  training  must  be  carried  out  using  a 
rigorous,  well-accepted  methodology.  The  "receiver-operating-characteristic"  (ROC)  graph 
(Green  and  Swets,  1966)  provides  such  a  technique.  The  ROC  graph  is  based  upon  a  2  x 
2  stimulus-response  matrix  such  as  pictured  in  Figure  4.1  in  which  the  known  stimuli  are 
row  entries  and  the  perceived  response  options  are  column  outputs.  The  capital  letters  (A, 
B,  C,  and  D)  represent  the  total  number  of  times  each  type  of  event  occurred.  For 
example,  A  is  the  total  number  of  times  a  lightning  stroke  occurred  in  a  specific  tile  in  a 
specific  time  frame  as  it  was  predicted  to  occur.  The  conditional  probabilities  (e.g.  P(Yly)) 
are  shown  underneath  these  total  numbers  of  specific  occurrences  given  by  the  names 
underneath  the  conditional  probabilities. 
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P(Yly)  +  P(Nly)  =  1  and  P(Yln)  +  P(Nln)  =  1 


|  Figure  4.1  Stimulus- Response  or  'Contingency'  Matrix  for  Evaluating  Networks 


The  upper  left-hand  box  gives  the  conditional  probability  of  the  network  responding 
that  lightning  will  occur  when  it  actually  occurred.  The  lower  right-hand  box  is  the 
conditional  probability  of  the  network  responding  that  lightning  will  not  occur  in  a  situation 
in  which  it  indeed  did  not  occur.  A  good  lightning  predictor  would  have  high  probabilities 
in  these  two  boxes.  The  lower  left-hand  box  is  the  probability  that  the  network  would 
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predict  lightning  when  it  did  not  occur  -  the  probability  of  a  false  alarm  (PFA).  The  upper 
right-hand  box  is  the  probability  of  predicting  no  lightning  when  it  did  occur. 

A  decision  evaluation  system  can  be  built  on  conditional  probabilities.  P(Y!y),  the 
probability  of  the  system  predicting  Y  given  that  y  is  observed.  The  probability  that  the 
decision  system  response  is  Y  to  weather  stimuli,  y,  applies  to  the  period,  t  to  t+dt,  and 
some  specified  area,  da.  If  Y  is  not  the  response  we  must  choose  N.  For  this  reason  the 
two  equations  hold: 


P(Yly)  +  P(Nly)  =  1  and  P(Yln)  +  P(Nln)  =1  (4.1) 

This  allows  one  to  plot  all  the  consequences  of  the  decision  making  on  a  two  degree  of 
freedom  graph,  the  ROC  graph  which  is  pictured  in  Figure  4.2. 

In  evaluating  lightning  detection  networks  the  behavior  of  the  output  nodes  is  used 

as  a  scoring  measure.  An  output  node  'activation'  level,  a,  will  vary  from  0.1  (event 
predicted  not  to  happen)  to  0.9  (event  predicted  to  happen).  A  threshold  T  has  been 
adopted  for  decision  making.  If  the  activation  level  of  an  output  node  is  greater  than  T,  a  > 

T,  the  positive  prediction  Y  is  declared;  otherwise,  T  >  a,  a  negative  prediction  N  is 
declared.  When  evaluating  the  network,  all  predictions  are  compared  with  the  known 
results  (i.e.  observations)  y  and  n  in  a  contingency  matrix  (see  Figure  4.1)  as  T  varies. 

Even  in  high  activity  regions,  lightning  strikes  are  rare.  It  is  important  for  the 
prediction  scheme  to  correctly  predict  non-lightning  events  which  numerically  dominate 
stroke  'events'.  A  lightning  predictor  would  have  a  high  overall  correct  percentage  if  it 
always  predicted  'no  strike'.  This  is  the  'play  the  winner  strategy'  (Stanski  et  al.,  1989). 
The  successful  predictor  must  predict  the  numbers  of  lightning  events.  So  while  a  low 
false  alarm  rate  is  desirable,  it  is  not  indicative  of  predictive  power  when  true  alarms  are 
rare. 


Measures  of  performance  are  adapted  based  on  the  following  definitions  taken  from 
the  contingency  matrix  of  Figure  4.1  (Stanski  et  al.,  1989)  from  which  the  following 


performance  measures  can  be  defined, 

Probability  of  Detection,  PoD  =  A/(A  +  B)  (4.2) 

Probability  of  False  Alarm,  PFA  =  C/(C  +  D)  (4.3) 

Miss  or  Probability  of  False  Dismissal,  PFS  =  B/(A  +  B)  (4.4) 

False  Alarm  Ratio,  FAR  =  C/(A  +  C)  (4.5) 

Correct  Rejection  =  D/(C  +  D)  (4.6) 


The  difference  between  the  Probability  of  False  Alarm  PFA  defined  in  Eq.  (4.3) 
and  the  False  Alarm  Ratio  FAR  in  Eq.  (4.5)  is  significant.  The  PFA  is  the  conditional 
probability  of  (false)  positive  prediction  given  that  the  event  was  not  observed.  On  the 
other  hand,  the  FAR  is  the  ratio  of  false  predictions  of  the  event  to  all  predictions  of  the 
event.  The  FAR  measure  is  only  sensitive  to  predictions  and  is  a  good  measure  of  the 
system  performance  only  when  false  alarms  are  very  undesirable.  Chi  the  other  hand,  the 
PFA  is  a  better  measure  of  the  likelihood  of  the  actual  events  and  should  be  used  for 
decision  making. 
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Two  success  and  skill  measures  are  used.  The  Critical  Success  Index  (CSI) 
introduced  by  Donaldson,  et  al.,  1975,  is  a  measure  of  relative  accuracy  since  it  shows  the 
ratio  of  predicted  cases  vs.  both  predicted  and/or  observed  cases, 

CSI  =  A/(A  +  B  +  C)  (4.7) 

The  range  of  CSI  values  extends  from  0  to  1.  CSI  differs  from  FAR  and  POD  because  it  is 
sensitive  to  both  false  alarms  and  missed  events.  To  evaluate  the  performance  of  the 
network  the  Heidke  skill  score  is  used  (Stanski  et  al.,  1989).  The  Heidke  skill  score  values 
range  from  -»  to  +1.  Plus  one  indicates  the  perfect  score  and  zero  means  that  the  number 
of  correct  predictions  equals  the  number  of  expected  correct  predictions.  Thus  the  skill 
"subtracts"  the  number  of  correct  predictions  which  would  have  been  obtained  merely  by 
chance. 


Skill  =  [A  +  D  -  (Yy  +  Nn)/M]/[M  -  (Yy  +  Nn)/M]  (4.8) 

where  M  is  Total  Number  of  Observations: 

M  =  A  +  B  +  C  +  D  (4.9) 

Y  =  A  +  C  (4.10) 

N  =  B  +  D  (4.11) 

y  =  A  +  B  (4.12) 

n  =  C  +  D  (4.13) 


The  values  in  the  contingency  table  of  Figure  4.1  are  used  to  compute  conditional 
probabilities  in  a  receiver  operating  characteristic  (ROC)  graph  illustrated  in  Figure  4.2. 
Overall  network  performance  depends  on  the  value  of  the  threshold,  T,  chosen  by  the  user. 
If  T  is  large  then  fewer  events  will  be  classified  as  lightning  strikes,  decreasing  PoD  and 
PFA  at  the  same  time.  That  would  be  indicated  by  a  point  in  the  lower  left  side  of  the  ROC 
graph  of  Figure  4.2.  Small  T  will  lead  to  high  false  alarm  rates  in  the  upper  right-hand 
corner  of  the  graph.  For  a  given  network  the  ROC  is  a  curve  between  these  two  extremes. 
Setting  T  is  the  responsibility  of  the  user  who  must  balance  the  benefit  of  a  correct 
prediction  against  the  cost  of  a  mistake. 

When  the  probability  of  detection  is  equal  to  the  probability  of  a  false  alarm  the 
decision-making  process  is  doing  no  better  than  it  would  by  guessing.  This  situation  is 
shown  by  the  straight  line  c  in  Figure  4.2.  For  a  lightning  prediction  system,  one  expects 
that  curve  c  would  be  the  performance  of  an  untrained  network.  Increasing  levels  of 
training  are  shown  by  the  curves  a  &  b  which  move  toward  the  upper  left-hand  comer  of 
the  graph.  For  a  given  level  of  training  and  a  series  of  threshold  values  (Tj,  T2,  and  T3), 
one  can  estimate  the  ROC  curve  as  shown  by  b  in  Figure  4.2.  The  integral  under  the  ROC 
(the  shaded  area)  is  a  measure  of  the  network's  performance.  For  c  the  shaded  area  is  zero 
corresponding  to  chance  predictions  or  'shooting  in  the  dark'.  To  find  an  optimal 
threshold,  To,  the  user  must  estimate  the  benefits  and  losses  of  correct  and  incorrect 
predictions,  respectively.  Having  trained  a  network,  one  adjusts  thresholds  until  the  best 
balance  is  struck  between  these  benefits  and  losses.  An  important  ANS  feature  is  that,  with 
the  resumption  of  training,  skill  level  might  be  incrementally  increased  at  a  local 
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observation  site.  It  is  important  to  consider  the  overall  process  of  retraining  and  resetting 
thresholds  in  the  ultimate  local  user's  interface  to  the  processor. 

The  scoring  of  the  present  lightning  predictor  contains  some  elements  adapted  to  the 
geography  and  future  time  intervals  of  interest  for  KSC.  The  geographical  breakdown  for 
KSC  is  shown  in  Figure  3.4.  For  each  geographical  subdivision,  the  network  has  been  set 
up  to  predict  lightning  activity  in  four  future  time  windows.  Prediction  for  the  interval 
from  the  present  to  15  minutes  in  the  future  is  called  the  ’nowcast.'  Predictions  for  the 
interval  from  15  minutes  to  30  minutes  is  called  'Time  1/2’;  between  30  minutes  and  1  hour 
is  'Time  T;  and  between  1  hour  and  2  hours  is  Time  2'.  Statistics  for  each  prediction 
window  are  separately  compared  with  each  time  in  the  future. 


1.0 


PoD 


Tl 


0.0 

0.0  PFA  1  -0 


Figure  4.2  ROC  Graph  Showing  the  Predictor's  Performance  at  Several  Thresholds  (Tj, 
T2,  and  T3)  at  one  Level  of  ANS  Training,  b,  among  Several  Levels  (a,  b,  and  c). 


The  scoring  system  is  conservative.  In  the  event  that  the  network  predicts  lightning 
activity  for  'Time  1/2'  and  no  lightning  stroke  occurs  in  the  'Time  1/2'  window  the 
network  scores  zero..  This  is  so  even  if  subsequently  a  lightning  stroke  occurs  for  that 
geographical  region  at  'Time  1'.  Generally,  the  scoring  gives  the  predictor  lower 
performance  scores  than  were  it  credited  with  subsequently  observed  strikes  for  any  of  its 
time  windows  as  it  is  penalized  for  predictions  which  are  'too  early'  or  'too  late.' 
Predictions  for  specific  time  intervals  and  tiles  are  used  to  construct  contingency  tables  as 
the  basis  for  comparison  with  alternative  lightning  prediction  systems. 

4.2.  Predictor  Performance 

The  literature  on  the  performance  of  mathematical  correlation  schemes  for  predicting 
lightning  includes  the  work  of  Watson  and  Blanchard  (1984)  which  associates  lightning 
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with  wind  convergence.  A  quantitative  method  of  Watson  et  al.  (1987)  is  used  in  this 
study.  Essentially,  a  lightning  strike  is  predicted  when  wind  divergence  is  less  than  -1.5  x 
10'4  sec'l  for  at  least  10  minutes  in  the  KSC  area.  This  criterion  was  used  by  Watson  et 
al.  (1987)  to  create  a  contingency  table  and  will  be  used  as  a  benchmark  forjudging  the 
results  obtained  here  using  ANS  lightning  strike  predictors. 

The  lightning  strike  predictions  of  this  study  are  compared  with  the  results  of 
Watson  et  al.  (1987;  in  Table  4.3  expressed  in  the  scoring  parameters  Section  4.1. 
Lightning  prediction  using  ANS  is  shown  to  reach  a  performance  level  comparable  to  that 
obtained  by  Watson  et  al.  In  addition,  it  is  important  to  establish  that  the  assumptions  and 
measurement  conditions  of  the  Watson  et  al.  (1987)  study  and  this  study  are  sufficiently 
similar.  This  comparison  is  shown  in  Table  4.4  where  it  is  seen  that  the  parameters  of  the 
two  studies  are  sufficiently  alike.  In  conclusion,  the  predictions  obtained  using  trained 
artificial  ANS  (as  a  means  for  building  a  lightning  flash  predictor)  and  the  predictions 
provided  by  the  state  of  the  art  are  close  in  overall  performance  measures. 

Table  4,3  Comparison  of  the  Results  of  Watson,  et  al.  ( 1987>  and  Those  of  this  Study. 


Performance  Measure 

This  Study 

Watson,  et  al. 

(1987) 

Probability  of  Detection 

0.41 

0.47 

Probability  of  False  Alarm 

0.57 

0.56 

Critical  Success  Index 

0.26 

0.29 

Table  4,4  Similarities  and  piffcrgnces.-of.tiie T_wQ.5tudi.es 


mmwEmm 

Watson  et  al.(I987) 

This  Study 

Period  Studied 

42  Days  in  1983 

7/24,25/88 

Prediction  Area 

Mesonet 

16  Tiles 

21x21  sq.  nmi. 

Wind  Sensors  Used 

54  feet 

All 

Data  Period 

5  minute 

5  minute 

Predicted  Value 

c-g  flashes  (LLP) 

c-g  flashes  (LLP) 

Prediction  Correr>  If 

Convergence  precedes  flash 

Flash  time  and  tile  correct 

The  reader  may  be  interested  in  noting  the  steps  in  improvement  of  one  of  the 
artificial  neural  systems  toward  the  performance  figures  seen  above  in  Table  4.3.  The 
results  of  Table  4.3  were  obtained  using  one  day’s  raw  wind  speed  and  direction  data  as 
training  input  to  the  networks  and  a  different  day's  wind  data  for  testing  the  network's 
predictions.  Shown  in  figure  4.4  is  a  ROC  plot  of  the  Probability  of  Detection  vs.  False 
Alarm  Rates  for  predicting  lightning  using  networks  trained  and  tested  on  the  same  and  on 
different  lightning  days.  The  open  symbols  are  for  incompletely  trained  networks  (pass 
error  decreasing  rapidly)  while  filled  symbols  are  for  completely  trained  networks  (pass 
error  decreases  slightly  with  further  training).  As  the  data  considered  are  located  in  the 
lower  left  hand  side  of  the  ROC  plot  only  part  of  the  plot  is  shown.  One  result  is  that  the 
steep  (at  45  degrees)  straight  line  c  of  Figure  4.2  is  shown  by  the  slightly  sloped  straight 
line  in  the  lower  part  of  Figure  4.4. 
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The  training  behavior  reveals  several  aspects  of  using  an  ANS  for  prediction. 
Consider  a  network  trained  on  one  day  and  tested  on  another  day's  data  (triangular 
symbols).  Performance  improvement  is  seen  in  the  upward  movement  of  the  PoD  from 
'slight'  training  (open  symbols)  to  'full'  training  (black  symbols)  in  Figure  4.4  while  PFA 
increases  proportionately  less  (the  black  ramp  at  the  bottom  of  the  figure  is  for  equal  rates 
of  increase  in  PoD  and  PFA).  It  is  expected  that  as  more  is  learned  about  building  lightning 
predictors  using  ANS  more  performance  improvements  will  be  realized. 

The  apparently  better  performance  in  Figure  4.4  when  training  and  testing  on  the 
same  day's  data  (square  symbols)  results  from  an  unacceptable  procedure.  A  single  day's 
meteorological  data  contain  a  certain  amount  of  idiosyncratic  events  (e.g.  turbulence)  that 
do  not  represent  the  underlying  behavior  of  lightning  strikes.  To  eliminate  such  an  illusion 
of  good  performance,  ANS  predictors  must  always  be  tested  on  different  data.  At  present 
the  'predictive  parameters’  of  Section  2.2.1  are  not  used  as  ANS  inputs  requiring  that  the 
training  and  testing  data  be  taken  from  similar  'flow  regimes'  and  so  forth.  Eventually, 
ANS  predictors  will  account  for  these  conditioning  effects  automatically  in  the  inputs. 


Figure  4.4  Improvements  in  ANS  Lightning  Predictor  Performance  as  Training  Time  is 
Increased.  Testing  Across  Different  Days  has  the  Effect  of  Testing  for  ANS  Networks 
which  Exhibit  Some  Valid  Mapping  of  the  Underlying  Correlations  Between  the  Chosen 
Inputs  and  the  LLPWS  Target  Values. _ _ 


It  is  expected  that  significant  further  improvement  in  predictor  performance  can  be 
obtained.  Synthesis  of  different  types  of  data  is  one  of  the  strengths  of  the  ANS  approach. 
Network  predictive  power  can  be  expected  to  increase  when  other  inputs  (e.g.  electric  field, 
wind  divergence,  temperature,  humidity  and  radar  returns)  are  included  in  the  input  array. 
Predictive  power  is  also  expected  to  improve  when  more  than  one  day’s  data  are  used  in 
training.  Recommendations  for  improving  lightning  predictors  are  given  in  Section  5. 
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L _ Conclusions 


This  study  investigated  the  possibility  of  using  field  data  for  specifying  and  training 
an  ANS  which  can  be  used  to  forecast  severe  weather  events,  specifically  the  occurrence  of 
lightning  strikes.  Comparisons  have  been  made  with  earlier  methods  based  upon 
correlation  of  wind  divergence  with  the  later  occurrence  of  a  lightning  flash.  Using  KSC 
meteorological  data  for  wind  and  lightning  strikes  a  state-of-the-art  skill  level  prediction 
performance  has  been  attained.  Given  the  complexity  of  lightning  strikes,  the  current 
results  should  be  understood  as  indicative  of  a  promising  initiative  and  cannot  be 
considered  definitive.  Thus,  recommendations  are  made  for  further  investigation. 

5.1.  Accomplishments 

Training  ANS  for  forecasting  severe  weather  events  has  required  that  a  series  of 
steps  be  taken  to  present  a  reasonably  rigorous  evaluation  of  the  program  goals.  The  main 
subtasks  undertaken  and  completed  are: 

(1)  Data  Reformatting  -  Reformatting  massive  weather  data  sets  for  input  to 
ANS  has  been  carried  out.  The  data  sets  on  winds,  electric  fields,  and  lightning 
strokes  were  supplied  by  KSC  personnel  The  reformatting  software  was 
developed  at  KTAADN,  INC. 

(2)  Predictor  Performance  Evaluation  -  A  predictor  performance 
evaluation  scheme  was  created  based  upon  well-accepted  procedures  useful  for 
evaluating  decision  making  processes. 

(3)  Trial  Networks  -  A  family  of  ANS  was  created  and  adapted  to  the 
geography  of  and  relevant  time  scales  for  KSC. 

(4)  Storm  Physics  -  Consideration  was  given  to  the  physics  of  severe  storms 
and  lightning  as  a  basis  for  constructing  prediction  networks. 

(5)  User  Interface  -  A  user  interface  for  invoking  the  needed  items  and 
functions  for  building  ANS  was  devised.  This  interface  provided  the  following 
functions:  (i)  entering  reformatted  weather  data  sets,  (ii)  constructing  and 
subsequently  training  ANS  predictors,  (iii)  evaluating  the  network  skill  levels 
using  accepted  comparison  techniques. 

(6)  Increased  Skill  Levels  -  Training  of  ANS  to  significant  skill  levels 
was  carried  out  for  the  prediction  of  lightning  based  upon  wind  fields.  The 
networks  trained  in  this  project  delivered  a  prediction  performance  which 
compared  favorably  with  the  performance  described  by  Watson  et  al.  (1987). 

(8)  Incrementally  Retrainable  Weather  Event  Predictor  -  The 

training  of  the  networks  has  been  stopped  and  restarted  with  no  observable  loss 
in  the  previous  level  of  network  performance.  This  feature  is  a  major  advance 
over  procedurally  programmed  predictors  which  often  require  fundamental 
reprogramming  when  improvements  are  integrated  into  the  software.  Using  the 
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user  interface  developed  in  this  study,  a  meteorologist  could  resume  training  of 
the  predictor  by  introducing  a  new  data  set  without  needing  to  know  a  high  level 
programming  language  (such  as  FORTRAN.) 

(9)  Turnkey  Macintosh  IIx  Predictor  Workstation  with  User 

Interface  and  Data  Files  -  A  prototype  operating  lightning  predictor 
workstation  has  been  built  with  basic  functionalities  in  place. 

In  sum,  this  study  has  provided  the  following  components  of  an  ANS-based 
'LightningLynx©'  computer  man/machine  interface  program  for  training  lightning 
predictors  in  the  KSC  environment: 

A)  Documentation 

i)  Final  report 

B)  LightningLynx©  Software 

i)  LightningLynx©  interface  software  (source  code) 

ii)  Backpropagation  learning  run  time  library  (machine  code  only) 

iii)  CyberMac  and  LLP  Plotter  software  developed  during  the  contract  period  for 
reformatting  KSC  data  tapes 

iv)  Data  and  network  documents  created  during  the  contract 

C)  Hardware 

i)  Macintosh  IIx  computer 

ii)  Apple  Color  Monitor  with  stand 

iii)  Qualstar  model  1054  9-track  tape  transport 

iv)  Keyboard,  mouse  and  cables  for  items  1-3 

5.2.  Recommendations 

This  examination  of  the  use  of  ANS  for  predicting  weather  events  recommends 
several  major  areas  for  research  and  development, 

(A)  Use  of  Larger  Data  Sets  for  Improving  ANS  Predictor  Performance 

-  Larger  data  sets  of  single  meteorological  inputs  would  be  examined.  This  task 
would  investigate  ANS  trained  with  large  data  sets: 

(a)  Larger  wind  field  and  LLP  data  sets 

(b)  Larger  electric  field  and  LLP  data  sets 

(B)  Use  of  New  Combinations  of  Data  Sets  and  Collateral  Data  for 

Improving  ANS  Predictor  Performance  -  Watson  et  al.  (1988)  state  that 

"Only  through  the  integration  of  all  the  data  bases  available  can  the  forecaster  ► 

make  the  best  possible  prediction."  This  task  would  address  ANS  networks 

which  provide  for  combining  different  meteorological  data  sets.  Both  ANS 

layouts  which  fuse  different  types  of  inputs  and  ANS  training  using  combined 

sets  of  field  data  would  be  carried  out.  Issues  addressed  in  this  task  would 

improve  performance  by  better  utilizing: 

(a)  Optimum  ANS  layouts  for  fusing  meteorological  data 
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(b)  ANS  predictors  trained  using  wind  and  electric  field  inputs  with  LLP  data 

(C)  Use  of  the  Physics  of  Storms  for  Improving  ANS  Predictor 
Performance  -  This  task  would  consider  how  new  ANS  predictors  might  be 
attained  using  knowledge  of  the  physics  of  storms  regardless  of  the  current 
availability  of  data.  Issues  addressed  in  this  task  would  be: 

(a)  Multi-tiered  ANS  layouts 

(b)  Relative  value  of  currently  unused  types  of  physical  measurements 

(D)  Development  of  Processes  for  Routinely  Evaluating  Predictor 
Performance  -  For  the  ranking  trained  ANS  predictors  this  task  would 
implement  a  range  of  evaluation  processes  in  the  User  Interface.  Issues 
addressed  in  this  task  would  include: 

(a)  Accuracy  or  Skill  levels 

(b)  Automatic  testing 

(E)  Investigation  of  Available  Real-Time  Processors  for  an  ANS 

Lightning  Predictor  -  This  task  would  survey  real-time  hardware  (e.g. 
transputer  boards)  that  could  support  a  viable  capability  at  a  remote  site.  Issues 
addressed  in  this  task  will  include1: 

(a)  Accelerated  workstations  (e.g.  Macintosh™  IIx  with  transputer  boards) 

(b)  Industrial  process  control  machines  (e.g.  PIM™) 

(F)  Develop  LightningLynx©  User  Interface  in  Conjunction  with  KSC 

and  Meteorologists  -  The  task  would  extend  and  test  the  User  Interface 
functionality.  This  task  would  address  a  variety  of  candidates: 

(a)  Help  menus, 

(b)  variation  of  user  imposed  sensor  thresholds, 

(c)  'strip  chart'  display  of  recent  sensor  data 

(d)  implement  visible/audible  alarm  when  prediction  threshold  is  exceeded. 

(e)  Alternative  ANS  types 

(G)  Cost/Benefit  Ratios  vs.  Decision  Thresholds  at  KSC  -  This  task  would 

construct  complete  ROC  graphs  by  varying  decision  thresholds.  This  task  will 
investigate  several  issues: 

(a)  Alternative  means  for  setting  the  most  useful  thresholds 

(b)  Alternative  ways  of  presenting  cost/benefits  to  KSC  decision  makers 

(c)  Effects  of  meteorology  on  optimal  thresholds  (i.e.  weather  regime,  etc.) 

The  goal  of  the  research  and  development  program  outlined  above  is  the  development  of  a 
robust,  reliable  lightning  prediction  system  prototype  which  can  be  upgraded  directly  by 
meteorologists  at  remote  sites  using  locally  collected  data. 


1  PIM  is  a  registered  trademark  of  the  Parallel  Inference  Machine  of  Flavors  Technology  of  Amherst,  NH 
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